Problems

The project's entire migration history lived in a single db/migrations.ts (previously db/index.ts) file as a sequence of numbered functions (migrateModelsV1 through migrateModelsV24). This caused several compounding problems as the project grew:

• Contributor conflicts: Two contributors working on separate features both needed to add migrateModelsV25. Even when their actual changes had nothing to do with each other, they were guaranteed a merge conflict or inconsistency in sequence on the function name and the call site in migrateDbSchema. Resolving it required manually renumbering and re-testing.

• No way to undo local changes: When contributor B had written and applied a local migration, and contributor A's work landed on main first, B had no clean recovery path. The only options were manually writing compensating SQL directly in the SQLite file, or deleting the entire DB and losing all local API keys, settings, and request history. There was no down concept at all.

• Migration file becoming too large: It's becoming increasingly harder to maintain.

As the project grows and contributors work asynchronously, these problems make DB changes increasingly painful to coordinate.

Description of changes

• add a file-per-migration system with up/down support
• track applied migrations in a dedicated migrations table
• First migration file, 20260101_000000_legacy_baseline.ts has changes in DB up until this PR.
• five new CLI commands
  • npm run db:migration:create --name=<new migration file name> (scaffold a new migration file)
  • npm run db:migration:up (up to latest migration)
  • npm run db:migration:down (roll back the latest migration)
  • npm run db:migration:status (migration status)
  • npm run db:migration:fresh (init fresh db; clear all data)

Test

• add round-trip CI test (up → down to baseline → up, schema must match)

npm run test:migrations   # round-trip up/down/up test (also runs in CI)
npm test                  # full test suite

How to use

Applying migrations (first-time dev setup or after pulling changes that include new migrations in dev env):

i.e. when NODE_ENV=development, otherwise migrations run automatically to the latest

npm run db:migration:up

The contributor B conflict workflow:

# You have local migration 20260615_143022_my_feature.ts applied
# Contributor A's 20260614_090000_their_feature.ts just merged to main
# in Contributor B's branch

npm run db:migration:down     # undo your local migration
git checkout main
git pull origin main
npm run db:migration:up       # applies A's migration
git checkout <contributor B's branch>

# adjust your migration if there's a genuine conflict

npm run db:migration:up

How to create, write and up migration files dev workflow

Create a migration file

npm run db:migration:create --name=add_providers

It will create a new migration file in ./server/src/db/migrations

Edit the scaffolding

// up: creates the table
export function up(db: Database.Database): void {
  db.exec(`
    CREATE TABLE IF NOT EXISTS providers (
      id   INTEGER PRIMARY KEY AUTOINCREMENT,
      name TEXT    NOT NULL UNIQUE
    )
  `);
}

// down: drops it
export function down(db: Database.Database): void {
  db.exec(`DROP TABLE IF EXISTS providers`);
}

Up the migration changes into DB

npm run db:migration:up

Dev Notes

• This is not a breaking change, existing users' data remain intact. And new database migrations would work fine.
• Any db changes merged into main should only be changed via forward migration/a new migration, not migration down.
• The setup is only for NODE_ENV=development, other env's should up to the latest migration automatically on boot.
• I have added a CONTRIBUTING.md, feel free to modify or remove it if necessary.
• README.md in server/.../db is a bit repetitive with CONTRIBUTING.md, so one can be removed as well for consistency.